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IN THE CLAIMS: 

1. (Currently amended) A method for operating a data, classifier , the method comprising: 

providing test input daia elements and corresponding test output data elements, 
providing the test input data elements to a data classifier to generate result output 
data elements, 

generating measures of difference based on differences between each test output data 
element and each corresponding result output data element, 

associating the measures of difference with categories corresponding to different 
vahies of measures of difference, and 

based on the number of measures of difference associated with the categories , 
generating a performance measure of the data classifier 

2. (Cancelled) 

3. (Currently amended) The method of claim i L wherein generating a performance 
measure further includes: 

associating the categories with weights, 

for each category, generating a product based on the weight and the number of 
measures of difference associated with the category, and 
generating a sum of the products. 

4. (Currently amended) The method of claim 3, wherein associating the categories 
includes: 

associating the categories with weights based on the values of measures of difference 
associated with the categories. 
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5. (Currently amended) The method of claim 3, wherein associating the categories 
includes: 

associating categories having larger values of measures of difference with greater 
weights than categories having smaller values of measures of difference. 

6. (Original) The method of claim 3, wherein generating a performance measure further 
includes: 

normalizing the sura of the products. 

7. (Original) The method of claim 6, wherein normalizing includes: 

normalizing the sum of the products based on one of: the number of test input data 
elements arid a reciprocal of a logarithm of the number of test input data elements. 

8. (Original) The method of claim I, further comprising: 

training the data classifier, and 

based on comparing the performance measure with a threshold, retraining the data 
classifier. 

9. (Original) The method of claim 1, wherein the test data are based on 
telecommunications data. 

10. (Original) The method of claim 1, wherein the data classifier includes a neural network. 

11. -16. (Cancelled) 

17. (Currently amended) A processor program computer product for operating a data 
classifier, the proooaaor program computer product disposed on a proc e ssor computer- 
readable medium and comprising instructions to cause a processor to: 
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receive test data including test input data elements and corresponding test output data 
elements, 

receive result output data elements generated by the data classifier based on the test 
input data elements, 

generate measures of difference based on differences between each test output data 
element and each corresponding result output data element, asd^ 

associate the mea sures of difference with categories correspondine to different 
values of measures of difference, and 

based on the number of measures of difference associated with the categories , 
generate a performance measure of the data classifier. 

18. (Cancelled) 

.19. (Currently amended) The processor program computer product of claim 4& 17, wherein 
the instructions to generate a performance measure further include instructions to cause the 
processor to : 

associate the categories with weights, 

for each category, generate a product based on the weight and the number of 
measures of difference associated with the category, and 
generate a sum of the products, 

20. (Currently amended) The processor - program computer product of claim 19, wherein 
the instructions to generate a performance measure further include instructions to cause the 
processor to : 

normalize the sum of the products based on one of: the number of test input data 
elements and a reciprocal of a logarithm of the number of test input data elements. 

21. (Currently amended) The processor program computer product of claim 17, further 
comprising instructions to cause the processor to : 
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train the data classifier, and 

based on comparing the performance measure with a threshold, retrain the data 
classifier. 

22. (Currently amended) The procassor progra m computer product of claim 17, wherein 
the test data are based on telecommunications data. 

23. (New) A system for operating a data classifier, the system comprising: 

an input receiver operable to receive test data, including test input data elements and 
corresponding test output data elements, 

a data classifier operable to generate result output data elements in response to test 
input data elements, 

a difference generator operable to determine a measure of difference between each 
test output data element and each corresponding result output data element, 

a data association manager operable to associate the measures of difference with 
categories corresponding to different values of measures of difference, and 

a performance measure generator operable to generate a performance measure of the 
data classifier based on the number of measures of difference associated with the categories. 

24. (New) The system of claim 23, wherein the perfoimance measure generator is further 
operable to: 

associate the categories with weights, 

for each category, generate a product based on the weight and the number of 
measures of difference associated with the category, and 
generate a sum of the products. 

25. (New) The system of claim 24, wherein the performance measure generator is further 
operable to normalize the sum of the products based on one of: the number of test input data 
elements and a reciprocal of a logarithm of the number of test input data elements. 
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26. (New) The system of claim 23, the system further comprising a data classifier trainer 
operable to: 

train the data classifier, and 

based on comparing the performance measure with a threshold, retraining tie data 
classifier. 

27. (New) The system of claim 23, wherein the test data are based on telecommunications 
data. 
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